Skip to content

Profiling images can now be generated from crossbuild images#5834

Closed
marc-casavant wants to merge 2 commits into
FreeRADIUS:masterfrom
marc-casavant:new-profiling-image-build
Closed

Profiling images can now be generated from crossbuild images#5834
marc-casavant wants to merge 2 commits into
FreeRADIUS:masterfrom
marc-casavant:new-profiling-image-build

Conversation

@marc-casavant

@marc-casavant marc-casavant commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

Main changes:

  • Profiling images are built FROM <CROSSBULD_IMAGE>
  • Profiling images install useful profiling tools
  • Profiling images do not build FreeRADIUS

Remove stamp:
make crossbuild.ubuntu24.profile.reset

Generate the profiling Dockerfile.prof:
make crossbuild.ubuntu24.profile.regen

Build the profiling image:
make crossbuild.ubuntu24.profile.build

@mcnewton mcnewton left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few things to look at

Comment thread scripts/docker/crossbuild.mk Outdated
Comment thread scripts/docker/m4/profiling.deb.m4 Outdated
Comment thread scripts/docker/crossbuild.mk Outdated
Comment thread scripts/docker/crossbuild.mk Outdated
Comment thread scripts/docker/crossbuild.mk Outdated
Comment thread scripts/docker/docker.mk Outdated
@marc-casavant marc-casavant force-pushed the new-profiling-image-build branch 3 times, most recently from 21643c9 to 64edef6 Compare April 27, 2026 21:26
@arr2036 arr2036 self-requested a review April 29, 2026 17:05
@marc-casavant marc-casavant force-pushed the new-profiling-image-build branch 2 times, most recently from f8f3881 to ff45c05 Compare May 4, 2026 14:25
Comment thread scripts/docker/build/ubuntu24/Dockerfile.cb
Comment thread scripts/docker/build/ubuntu24/Dockerfile.prof Outdated
@marc-casavant marc-casavant force-pushed the new-profiling-image-build branch 2 times, most recently from c52eb04 to 7295e39 Compare May 14, 2026 19:32
@marc-casavant marc-casavant requested a review from arr2036 May 15, 2026 12:46
Comment thread scripts/docker/profiling/profiles/default-profiling/README.md Outdated
Comment thread scripts/docker/profiling/profiles/default-profiling/start.sh Outdated
Comment thread scripts/docker/crossbuild.mk Outdated
Comment thread scripts/docker/profiling/profiles/default-profiling/configure.sh Outdated
@marc-casavant marc-casavant force-pushed the new-profiling-image-build branch 2 times, most recently from 83477ce to a16694f Compare May 19, 2026 12:31
@arr2036

arr2036 commented May 19, 2026

Copy link
Copy Markdown
Member

@claude review in regards to the last 20 commits in master, and suggest fixes

@claude

claude Bot commented May 19, 2026

Copy link
Copy Markdown

Claude finished @arr2036's task in 5m 59s —— View job


Review

  • Read PR diff vs master
  • Examine the 20 recent master commits for context
  • Read changed files in detail
  • Identify issues and suggest fixes

Reviewed a16694f ("New profiling images based off of crossbuild images") in light of the 20 most-recent master commits — in particular the m4 refactor (24cddf6 factor common build setup into shared m4 snippets, add ci-base template), the pre-baked-image work (fe4b22c bake CI build environments into pre-baked base images), and the m4 path fix (d74e581 crossbuild: fix double-slash in m4 paths).


Critical

1. $APT_OPTS is undefined inside Dockerfile.prof. ARG APT_OPTS=... lives in crossbuild.deb.m4 (the base image build). profiling.deb.m4 starts a brand new build stage with FROM ${from} and never redeclares it, so in scripts/docker/m4/profiling.deb.m4:13,34,60,72,101 apt-get install -y $APT_OPTS ... expands to apt-get install -y — no --no-install-recommends, no --force-unsafe-io. The first RUN (kcachegrind + KDE/Qt libs) will pull in a huge recommended-package set as a result. Either redeclare ARG APT_OPTS=... after FROM, or drop the $APT_OPTS references and write the flags out explicitly. Fix this →

2. The makefile rules are emitted for every distro, but the m4 is Ubuntu-only. crossbuild.mk does $(foreach IMAGE,$(CB_IMAGES), $(eval ...)) which generates crossbuild.rocky9.profile.regen, crossbuild.debian12.profile.regen, etc. profiling.deb.m4 hard-codes ubuntu-dbgsym-keyring, ddebs.ubuntu.com, and t64-suffixed Ubuntu-24 dbgsym names (libreadline8t64-dbgsym, libssl3t64-dbgsym, …). Running the regen target for any non-Ubuntu-24 image produces a Dockerfile that will fail to build. Either:

  • Gate the rule expansion on a per-image whitelist (e.g. ifeq on D_NAME/PKG_TYPE), or
  • Add ifelse(D_NAME, ...) branches inside profiling.deb.m4 for rocky/debian (same pattern as the debian10/debiansid blocks in crossbuild.deb.m4).

3. PR doesn't use the new shared m4 snippets. Master commit 24cddf6 introduced common.apt.retries.m4, common.deb.toolchain.m4, common.deb.nr-extras.m4. profiling.deb.m4 duplicates apt-get update patterns and gets no retry handling — apt flakiness in CI will surface here. Compare ci-base.deb.m4:6-7 which simply does include('common.apt.retries.m4')dnl. Worth include-ing the same snippets here.


Significant

4. PROFILE_NAME is plumbed through m4 but never used. crossbuild.mk:291 passes -D PROFILE_NAME=$(PROFILE), but profiling.deb.m4 contains no PROFILE_NAME reference. Either drop the -D or use it (e.g. ENV PROFILE_NAME=PROFILE_NAME so runtime tooling can identify itself).

5. Generated header points at a non-existent make target. Dockerfile.prof:4 reads # Rebuild this file with make profiling.ubuntu24.regen, but no such target exists — the real target is crossbuild.ubuntu24.profile.regen. The header comes from Dockerfile.m4's make D_TYPE.D_NAME.regen placeholder, which assumes D_TYPE.D_NAME.regen is also the real target name. Either rename the makefile target to profiling.${1}.regen (top-level, sibling of crossbuild) or override the header for this template.

6. profile.build has no dependency on the base crossbuild stamp. crossbuild.mk:162 declares $(DD)/stamp-image.${1}-profile.build: $(DT)/${1}/Dockerfile.prof — but Dockerfile.prof does FROM freeradius40x-build/ubuntu24. On a clean checkout, make crossbuild.ubuntu24.profile.build will fail with image not found because nothing has built freeradius40x-build/ubuntu24 yet. Add $(DD)/stamp-image.${1} as a prerequisite. Fix this →

7. profile.reset doesn't remove the docker image. crossbuild.mk:298 only rm -fs the stamp. Without docker image rm freeradius4-$(PROFILE)/${1} the next profile.build reuses the cached image and the reset has no observable effect unless the user also passes NOCACHE=1. Either add the docker image rm, or split into profile.reset (stamp only) + profile.distclean (image too), mirroring the base image's reset vs distclean.

8. Inconsistent image-prefix naming. Base uses CB_IPREFIX:=freeradius40x-build. Profiling hard-codes freeradius4-$(PROFILE)/${1}. If freeradius4- is deliberate (it implies "FreeRADIUS 4.x"), introduce a PROF_IPREFIX:=freeradius4 (or derive from CB_IPREFIX) so the next major version doesn't require touching multiple places.


Minor / nits

9. Image bloat. cargo install inferno drags in the full Rust toolchain just to ship one binary; consider a multi-stage build that copies only the final /usr/local/bin/inferno-* out and discards cargo. Similarly, kcachegrind + KDE/Qt is hundreds of MB of GUI libraries inside what's logically a server container — these are typically used on a developer workstation reading exported callgrind output. Worth checking whether they need to live in the image at all.

10. Non-reproducible clones. git clone --depth 1 https://github.com/mheily/libkqueue.git and git clone --depth 1 https://github.com/brendangregg/FlameGraph aren't pinned. Pin to a release tag or commit SHA so two builds a month apart aren't subtly different. inferno --version 0.11.21 --locked already does this correctly.

11. rm -r /var/lib/apt/lists/* will fail on empty glob. Use rm -rf. Several other m4 files have the same pattern, so it's a pre-existing convention rather than a regression — but worth fixing here while you're touching the file.

12. Stamp name uses mixed separators. stamp-image.${1}-profile.build mixes - and .. Existing rules use stamp-image.${1}. stamp-image.${1}.profile (without the .build suffix) reads more like the rest of the codebase.

13. No CI workflow consumes these targets. This matches @alandekok's "developer convenience" intent (per the discussion on the help text), but it's worth confirming explicitly. If profiling images are ever wired into CI, follow the fe4b22c/24cddf6 pre-baked-image pattern rather than rebuilding the image inside every CI run.


Review-comment status

The thread cleanup from earlier rounds looks resolved:

  • default-profiling naming adopted ✓
  • profiles/default-profiling/{README,start,configure}.sh removed ✓
  • m4-path / filter profiling no longer needed since profiling lives outside build/ … but note: scripts/docker/build/ubuntu24/Dockerfile.prof is inside build/, which contradicts mcnewton's premise. Worth checking whether anything in docker.mk still globs build/*/Dockerfile* and would now pick up Dockerfile.prof unintentionally.
    • branch new-profiling-image-build

Comment on lines +157 to +166
#
# Build the profiling image
#
crossbuild.${1}.profile.build: $(DD)/stamp-image.${1}-profile.build

$(DD)/stamp-image.${1}-profile.build: $(DT)/${1}/Dockerfile.prof
${Q}echo "BUILD ${1} (freeradius4-$(PROFILE)/${1}) > $(DD)/build.${1}-profile.build"
${Q}docker build $(DOCKER_BUILD_OPTS) . \
-f $(DT)/${1}/Dockerfile.prof \
-t freeradius4-$(PROFILE)/${1} \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The new crossbuild.IMAGE.profile.build rule only depends on Dockerfile.prof, not on the base crossbuild image stamp $(DD)/stamp-image.${1}. Since Dockerfile.prof starts with FROM freeradius40x-build/<distro> (a local-only image tag), running make crossbuild.ubuntu24.profile.build on a clean checkout fails because docker tries to pull the non-existent image from a remote registry. Add $(DD)/stamp-image.${1} as a prerequisite of $(DD)/stamp-image.${1}-profile.build so the base image is built first.

Extended reasoning...

What the bug is

The rule introduced at scripts/docker/crossbuild.mk:160:

$(DD)/stamp-image.${1}-profile.build: $(DT)/${1}/Dockerfile.prof
        ${Q}echo "BUILD ${1} (freeradius4-$(PROFILE)/${1}) > $(DD)/build.${1}-profile.build"
        ${Q}docker build $(DOCKER_BUILD_OPTS) . \
                -f $(DT)/${1}/Dockerfile.prof \
                -t freeradius4-$(PROFILE)/${1} \
                >$(DD)/build.${1}-profile.build 2>&1
        ${Q}touch $(DD)/stamp-image.${1}-profile.build

declares only $(DT)/${1}/Dockerfile.prof as a prerequisite. However, the generated Dockerfile.prof (see scripts/docker/build/ubuntu24/Dockerfile.prof:6-7 and scripts/docker/m4/profiling.deb.m4:1-2) begins with:

ARG from=freeradius40x-build/ubuntu24
FROM ${from}

This is a local-only image tag, produced by the $(DD)/stamp-image.${1} rule. The docker build invocation passes no --build-arg to override from, so docker will look for freeradius40x-build/ubuntu24 locally and, on a clean checkout where this image has not been built, fall through to a registry pull which fails (e.g. pull access denied for freeradius40x-build/ubuntu24).

Why existing rules don't prevent it

Compare with the sibling docker.up.${1} rule a few lines below (around line 174):

$(DD)/docker.up.${1}: $(DD)/stamp-image.${1}

That rule correctly chains on the base image stamp, ensuring the base image is built first. The new profile.build rule simply omits this prerequisite.

Impact

The PR description explicitly advertises make crossbuild.ubuntu24.profile.build as a user-facing workflow, and it is listed in crossbuild.help at line 97. A developer following the PR description on a fresh checkout will hit a confusing failure because docker reports a pull error for what appears to be a local image name, rather than a clear "base image not built yet" message. CI is unaffected because it already runs the base crossbuild step first.

Step-by-step proof

  1. Fresh checkout of this branch — scripts/docker/crossbuild/stamp-image.ubuntu24 does not exist and the docker image freeradius40x-build/ubuntu24 is not present in the local docker daemon.
  2. Run make crossbuild.ubuntu24.profile.build.
  3. Make resolves: crossbuild.ubuntu24.profile.build$(DD)/stamp-image.ubuntu24-profile.build.
  4. The only prerequisite is $(DT)/ubuntu24/Dockerfile.prof, which already exists in the tree, so make proceeds directly to the docker build recipe.
  5. docker build -f scripts/docker/build/ubuntu24/Dockerfile.prof -t freeradius4-default-profiling/ubuntu24 . is invoked.
  6. Docker parses FROM ${from} with from=freeradius40x-build/ubuntu24. No local image with that tag exists, so docker attempts a registry pull.
  7. The pull fails with pull access denied for freeradius40x-build/ubuntu24, repository does not exist or may require 'docker login'. The build aborts and build.ubuntu24-profile.build records the failure.

Fix

Add $(DD)/stamp-image.${1} as a prerequisite of the profile build stamp:

$(DD)/stamp-image.${1}-profile.build: $(DD)/stamp-image.${1} $(DT)/${1}/Dockerfile.prof

This matches the convention used by $(DD)/docker.up.${1} and ensures the base crossbuild image exists before docker tries to consume it as the FROM of the profiling image.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added $(DD)/stamp-image.${1} as a dependency.

Comment thread scripts/docker/crossbuild.mk
Comment thread scripts/docker/build/ubuntu24/Dockerfile.prof
Comment thread scripts/docker/crossbuild.mk
Comment thread scripts/docker/build/ubuntu24/Dockerfile.prof
@marc-casavant marc-casavant force-pushed the new-profiling-image-build branch from f652b8d to f5b7e8b Compare May 19, 2026 17:51
@arr2036

arr2036 commented May 20, 2026

Copy link
Copy Markdown
Member

I've merged a set of changes derived from these, but with the duplication removed. There are now common macros which handled rendering docker files, and image lifecycles.

These are the new targets:

  - dockerfile.profiling — regen every Dockerfile.profiling
  - dockerfile.profiling.check — fail if any Dockerfile.profiling is stale
  - docker.profiling — build every profiling image
  - docker.profiling.up — docker run a sleep-forever container for every profiling image
  - docker.profiling.down — stop every profiling container
  - docker.profiling.reset — rm stamps for every profiling image
  - docker.profiling.clean — rmi + rm stamps for every profiling image (full nuke)

@arr2036 arr2036 closed this May 20, 2026
@arr2036

arr2036 commented May 20, 2026

Copy link
Copy Markdown
Member

There's also docker.profiling.IMAGE.VERB targets. Where image is one of debian12 debian13 debiansid rocky9 rocky10 ubuntu22 ubuntu24 ubuntu26

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants